Mixing Rates for the Alternating Gibbs Sampler over Restricted Boltzmann Machines and Friends
نویسنده
چکیده
Alternating Gibbs sampling is a modification of classical Gibbs sampling where several variables are simultaneously sampled from their joint conditional distribution. In this work, we investigate the mixing rate of alternating Gibbs sampling with a particular emphasis on Restricted Boltzmann Machines (RBMs) and variants.
منابع مشابه
Mixing Rates for the Gibbs Sampler over Restricted Boltzmann Machines
The mixing rate of a Markov chain (Xt)t=0 is the minimum number of steps before the distribution of Xt is close to its stationary distribution with respect to total variation distance. In this work, we give upper and lower bounds for the mixing rate of the Gibbs sampler over Restricted Boltzmann Machines.
متن کاملLayerwise Systematic Scan: Deep Boltzmann Machines and Beyond
For Markov chain Monte Carlo methods, one of the greatest discrepancies between theory and system is the scan order — while most theoretical development on the mixing time analysis deals with random updates, real-world systems are implemented with systematic scans. We bridge this gap for models that exhibit a bipartite structure, including, most notably, the Restricted/Deep Boltzmann Machine. T...
متن کاملBoltzmann machines and energy-based models
We review Boltzmann machines and energy-based models. A Boltzmann machine defines a probability distribution over binary-valued patterns. One can learn parameters of a Boltzmann machine via gradient based approaches in a way that log likelihood of data is increased. The gradient and Laplacian of a Boltzmann machine admit beautiful mathematical representations, although computing them is in gene...
متن کاملBayesian Mixtures of Bernoulli Distributions
The mixture of Bernoulli distributions [6] is a technique that is frequently used for the modeling of binary random vectors. They differ from (restricted) Boltzmann Machines in that they do not model the marginal distribution over the binary data space X as a product of (conditional) Bernoulli distributions, but as a weighted sum of Bernoulli distributions. Despite the non-identifiability of th...
متن کاملMaterial for : Factored Conditional Restricted Boltzmann Machines for Modeling Motion Style ∗ Graham
In this document, we provide additional details for variants of Conditional Restricted Boltzmann Machines (CRBMs). Specifically we focus on each of the four models compared in the Quantitative Evaluation (Sec. 4.4). We collect the formulae required for contrastive divergence learning of parameters, synthesis from a trained model by alternating Gibbs samping, and forward prediction from a traine...
متن کامل